AD74455 


AIR  UNIVERSITY 
UNITED  STATES  AIR  FORCE 


SCHOOL  OF  ENGINEERING 

'  ^  ' 

*v 

r  .  *  1 

WRIQHT-PATTERSON  AIR  FORCI  BASK/  OHIO 

-I...  I  I.. 

NATIONAL  TECHNICAL 

,  information  service 

II  *«  r.  porf.n.  MT  »>.  Cr.pmi.ffr 
'■  '-Htl.rM  V.\  .‘.’IM 


AF-Wr-O-OCT  «  3,500 


I 


1  y 


i&cussmEO _ 

Security  Classification  _ ^ ^ -««  ____ ^  ^  --i— -q—— 

I  *  DOCUMEN i  CONTROL  DATA  -  R &  D 

[  /5eCurl,fy  r'AmfiCdfion  of  title,  nodi  of  abitru*  f  and  indexing  annotatu^n  mi  f>t  entered  A<un  the  overall  repot  claasifte 

|i  ^  A  i  GINA  T  in  a  AC  Ti  vt  T  y  {Corporate  "author)  l^/».  REPO^T  s-ECuftn  •  CL«iS.HCATi< 


Air  Force  Institute  of  Technology  (AFIT-EN) 
fcirl ch t- Pat ts rsen  AFB,  Ohio  45433 


UT 

REPORT 

SECURH  ■  CL«i$,HCATlON 

UJi! 

CLASSIFIED 

Zb 

GROUP 

1 3  RFPOR1  TITcE 


A  $ystss  for  Validating  Hcssn  Reliability  Data 

<  OESCRiptive  NOTES  (T)‘pe  of  report  and  inclusive  da.es) 

AHT  Thesis _ 

5  au  THORlS)  (First  notTiC,  mddle  initial,  last  name) 


Edward  ls» 

Kajor  USAF 

ft  REPORT  DATE 

Rsces£cr  1971 

fta  ClNTRACT  OR  GRANT  NO 

b  PROJEC  T  NO 


10  DISTRIBUTION  STATEMENT 


7a.  TOTAL  NO  OP  PAGES 

77 

7b  NO  OP  REFS  ► 

27 

On  ORIG.NATOR'S  REPORT  NUM^ 

E  R<S) 

GRE/KATH/^“5 

Oh  OTHER  REPORT  noIS)  (Any  other  numbers  that  may  be  assigned 
this  report) 

Approved  for  public  release,  distribution  unlimited. 


release;  IAW  AFR  lDC^TT5 

Keith  A.  Williams,  1st  Lt. ,  USAF 

Acting  Director  of  Information _ 

13  ABSTRACT 


ONSORlNG  MILITARY  ACTIVITY 


Human  Resources  Laboratory 


The  iaasak  reii<sM!ity  data  (Data  Store,  £a  Indsx  ot  Electronic  Eijuijxsaef 
Operability)  darivad  by  the  Aserlcan  Institute  for  Research  was  us&d  to  select, 
codify,  or  construct  11  indicators  end  controls.  These  indicators  and  controls 
ware  esscdblod  into  too  panels  representing  the  best  and  worst  reliability  figures 
that  could  be  utilized.  To  produce  realistic  operational  panels,  little  relia¬ 
bility  difference  could  ba  sads  in  the  di  sans  ions.  The  pssal  oguiprsnt,  threap 
a  Tally  tape  reader,  m  operated  to  slEulste  sa  aircraft  flight  proff!®  free? 
tcJcc-off  to  level-off.  A  checklist  was  devised  varying  responds  control  indicator 
relationships.  Trainod  subjects  parforeed  discrete  actions  1b  rcspmsa  to  indi¬ 
cations  of  airspeed,  altitude,  and  vortical  velocity.  Their  res pecs©  was  recorded 
end  observed.  To  calnlraljs  e^uipsast  variability  for  higher  reliability  tasks, 
rocteadcat  error  recording  ©sd  interpretation  wss  perforrsd. 

Ia  addition  to  equipment  reliability  consideration,  easy  tangential  errors 
(such  as  proxlsfty  «d  siallaHky)  appeared  that  weald  affect  validation  and 
analysis.  A  near  emr  (called  response  error)  was  tabulated  for  a  pilot  sttfy 
psrfbrcad  oa  the  G^uipscat.  RalatJva  validation  of  the  data  was  accsspllshod. 


DD  ,FOoR.M473 


LflOASSIFiEP 


/A 


feacaa  RallebWIty  Bata 
Bata  St 97Q  Ir&x 


i 


,  / 

GRE/MATH/66-5 


A  SYSTEM  FOR  VALIDATING 
HUMAN  RELIABILITY  DATA 


THESIS 
Edward  Low 

Major  USAF 


Details  ot  illustrations  in 
this  document  may  be  better 
studied  on  microfiche 


Approved  for  public  release,  distribution  unlimited. 


A  SYSTEM  FOR  VALIDATING 


HUMAN  RELIABILITY  DATA 
THESIS 

Presented  to  the  Faculty  of  the  School  of  Engineering  of 
The  Air  Force  Institute  of  Technology 
Air  University 

in  Partial  Fulfillment  of  the 
Requirements  for  the  Degree  of 
Master  of  Science 


By 

Edward  Low 
Major  USAF 

Graduate  Reliability  Engineering 
December  1971 


Approved  for  public  release,  distribution  unlimited. 


I 


GRE/MATH/66-5 


Preface 


This  thesis  has  been  an  attempt  to  expand  the  overall  techniques 
for  quantification  and  validation  of  human  reliability  data.  It  was 
only  a  beginning,  but  one  with  colossal  potential.  It  was  a  complete 
thesis,  preceding  from  theory  through  design  and  building  of  equipment, 
designing  an  experiment,  testing  subjects,  tabulating  the  results,  and 
then  analyzing  them.  It  confirmed  the  hypothesis  that  testing  can  well 
become  more  a  function  of  the  testor  than  the  subject.  Many  error  factors 
emerged  that  should  not  be  ignored  in  serious  studies  of  complete  systems. 
The  response  error  developed  here  might  be  a  very  useful  tool  for  future 
analysis.  With  extreme  reliability  measurement,  redundancy  is  necessary 
to  control  the  reliability  of  complex  testing  equipment. 

I  am  indebted  to  Dr.  Donald  Topmiller  of  the  Aeromedical  Laboratory 
for  giving  me  this  interesting  and  challenging  thesis  topic.  The 
expectations  of  Dr.  Donald  Morris  imbued  me  with  a  sense  of  depth  and 
prespective  for  an  enormous  project.  Mrs.  Sara  Munger,  a  developer  of 
the  data,  gave  me  useful  guidance.  In  any  low-cost  project  manpower 
becomes  paramount  and  from  the  Human  Resources  Laboratory  Instrument 
Branch  with  which  I  worked,  John  Ferguson  helped  me  find  the  off-the- 
shelf  equipment  and  Noel  Schwartz  somehow  translated  ideas  into  electrical 
devices  and  put  this  conglomerate  mass  of  wires  and  relays  together. 

I  thank  Prof.  Moore  for  this  continued  interest  and  support  for 
me  and  my  class  and  Doris  Brown's  late  hours  of  typing. 

Edward  Low 
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Abstract 

The  human  reliability  data  (Data  Store,  An  Index  of  Electronic 
Equipment  Operability)  derived  by  the  American  Institute  for  Research 
was  used  to  select,  modify,  or  construct  11  indicators  and  controls. 
These  indicators  and  controls  were  assembled  into  two  panels  repre¬ 
senting  the  best  and  worst  reliability  figures  that  could  be  utilized. 
To  produce  realistic  operational  panels,  little  reliability  difference 
could  be  made  in  the  dimensions.  The  panel  equipment,  through  a  Tally 
tape  reader,  was  operated  to  simulate  an  aircraft  flight  profile 
from  take-off  to  level -off.  A  checklist  was  devised  varying  response 
control  indicator  relationships.  Trained  subjects  performed  discrete 
actions  in  response  to  indications  of  airspeed,  altitude,  and  vertical 
velocity.  Their  response  was  recorded  and  observed.  To  minimize 
equipment  variability  for  higher  reliability  tasks,  redundant  error 
recording  and  interpretation  was  performed. 

In  addition  to  equipment  reliability  considerations,  many 
tangential  errors  (such  as  proximity  and  similarity)  appeared  that 
would  affect  validation  and  analysis.  A  near  error  (called  response 
error)  was  tabulated  for  a  pilot  study  performed  on  the  equiDment. 
Relative  validation  of  the  data  was  accomplished. 
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A  SYSTEM  FOR  VALIDATING 
HUMAN  RELIABILITY  DATA 

I.  INTRODUCTION 


Purpose 

The  purpose  of  Inis  thesis  has  been  to  devise  a  system  for 
validating  human  reliability  data.  This  involved  the  design, 
construction,  and  test  of  an  apparatus  which  presented  an  operation 
using  indicators  and  controls  of  varying  dimensions  and  measuring 
the  reliability  of  the  operators.  The  roots  of  this  thesis  lie  in 
the  development  of  the  quantification  of  human  performance. 

Background 

Machines  that  man  operates  are  merely  extensions  of  his  abilities; 
yet  for  a  very  long  time  his  function  was  not  considered  a  part  of 
the  machine  output.  The  performance  parameters  of  the  man-machine 
system  evaluated  were  only  those  of  the  machine.  The  reliability  of  a 
system  was  generally  defined  in  terms  of  the  probability  of  system 
operation  without  equipment  breakdown.  With  the  advent  of  high  relia¬ 
bility  missiles  and  electronic  equipment,  the  human  factor  entered  a 
recognition  phase.  Shapero  (Ref  20)  pointed  out  that  20-53%  of  missile 
malfunction  reports  investigated  were  classified  as  human-initiated. 

Another  phase  of  human  element  assessment  might  be  called  the 
qualitative  phase.  Extensive  research  produced  a  large  number  of  te:<ts 
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and  research  reports  which  helped  engineers  design  equipment  for 
man's  use.  This  qualitative  data  gave  dimensions  and  equipment 
characteristics  for  optimum  human  performance.  Yet  a  need  existed  to 
quantify  the  human  role  if  the  total  mission  effectiveness  was  to  be 
determined. 

Quantification 

Irwin  (Ref  7)  outlined  several  approaches  to  the  quantification 
of  human  performance. 

1.  Analysis  of  field  experience. 

2.  Extrapolation  from  experimental  literature. 

3.  Conducting  special  studies  in  simulated  environment. 

4.  Conducting  special  studies  in  operational  environment,  and 

5.  Judging  and  rating  reliability. 

Basic  to  all  these  methods  is  a  reduction  of  a  mission  down  to 
a  specific  task  analysis. 

The  first  major  work  in  this  area  was  done  by  the  American 
Institute  for  Research  (AIR).  They  began  with  an  extensive  literature 
search.  Through  manipulation  of  the  data  and  their  own  judgement 
they  produced  reliability  figures  for  various  dimensions  of  simple 
indicators  and  controls  and  put  them  into  a  collection  called  Data 
Store. 

Rook  (Ref  17)  analyzed  error  data  for  23,000  production  defects 
detected  in  assembly  operations  of  electronic  equipment.  He  developed 
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a  classification  scheme  and  a  quantitative  model  for  evaluating  the 
contribution  of  human  error  to  the  degradation  of  product  quality. 

Swain  (Ref  21)  extended  Rook's  work  to  military  applications  and  used 
the  Data  Store.  His  method  was  called  Technique  for  Human  Error  Rate 
Prediction  (THERP).  He  used  a  probability  tree  to  cover  all  possible 
consequences  of  operator  action. 

A  computer  extension  of  this  methodology  was  developed  by  Miller 
(Ref  11).  He  oriented  his  approach  around  the  operator  of  U.S.  Army 
Signals  Corps  field  equipment.  Called  Tactical  Data  Entry  System 
(TACDEN),  it  was  basically  a  typewriter  type  of  keyboard  for  entry  of 
data,  a  magnetic  drum  for  storage  and  a  cathode-ray  tube  which  showed 
the  information  the  operator  was  supplying  the  system. 

AIR  Data  Store 

The  Data  Store  of  AIR  (an  Index  of  Electronic  Equipment  Operability) 
was  the  basic  building  block  for  many  approaches.  The  basic  unit  of 
evaluation  for  the  index  was  a  specific  step  or  action.  This  step  was 
subdivided  into  three  aspects:  (1)  inputs(stimuli  to  the  senses), 

(2)  mediating  processes  (thinking,  evaluating),  and  (3)  outputs  or 
responses  (motor  activities).  Equipment  behavior  components  were 
identified  that  were  likely  to  affect  each  aspect  of  behavior 
(Table  I).  The  authors  of  the  Data  Store  recognized  a  paucity  of 
categories,  mediating  process  and  a  need  for  materials  in  this  area. 

These  individual  components  were  then  broken  down  into  parameters 
and  then  into  dimensions  which  were  discrete  categories  or  intervals 
(Table  II). 
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Table  I 


List  of  Input,  Mediating  Process,  and  Qutput  Components 


Inputs 

Mediatina  Process 

.  .....  —  ..  ,u. — — 

Outputs 

Circular  Scales 

Identification/ Recognition 

Cable  Connections 

Counters 

Manipulation 

Cranks 

Labeling 

Disconnecting 

Lights 

Joysticks 

Linear  Scales 

1 

Knobs 

Non-Speech 

, 

Levers 

Scopes 

Object  PoSitipning 

Semi-Circular  Scales 

Pushbuttons 

Speech 

Rbtary  Selectors 

? 

Speech 

Toggle  Switches 

i 

Wri ti ng 

(From  Ref  17:11) 


Table  I, I 

i  i  • 

I 

Counter  Parameters  and  Dimensions 

1.  Size  (length) 

:  a.  1" 

b.  1-2" 

c-  3"  and  up  ' 

2.  Numbers  of.  drums  (or  digits)  ■ 

a.  1-3 
'  b,  4-5 

c.  7  ana  up 

3.  '  Style 

a.  Quantitative  reading 

b.  Qualitative  reading 


(From  Ref  15:4) 


4 


I 


GRE/MATH/66-5 


i 

Unfortunately,  the  Experimental  ,data  was  very  difficult  to  quantify 
and  large  gaps  existed.  Here  they  interpolated  and  extrapolated  from 

•  *  •  j 

related  studies,  used  their  judgement,  or  made  "quick  and  dirty" 
studies.  As  can  be  seen,  some  gaps  still  existed.  For  the  second 

I  i 

parameter  in  Table  II,  number  of'  drums,  the  quantity  6  is  emitted 
because  they  did  not  want  to  overinterpret  the  data. 

i  ,  I  • 

Over  a> variety  of  equipments  and  missions  the  range  of  operator 
reliabilities  was  85  to  90%.  This  was  interpreted  to  mean  that  10-15%  . 
of  the  time  an  operator  error  would  fail  or  seriously  degrade  mission 
effectiveness.  Since  no  field  tests  for  reliability  for  individual 

i  : 

task  steps  or  behavior  components  were  available,  they  derived  their 

:  !  I 

own.  Using  their  mean  unreliability  -figure  of  O'.  13  (10-15%),  they 
!  .1  .1 
divided  it  by  the  meari  number  of  steps  in  a  mission.  For  26  different 

equipments  the  average  number  of  s teps  was , 50 ,  so  their  mean  unrelia¬ 
bility  per  step  was  , 

,  0.13  =  0.0026  ' 

50  ■ 

This  means  t^iat  in  10,000  operation^  of  this  step,'  operator  error  would 

•  i 

fail  or  seriously  degrade  a  nrssi'-r,  26  times.  This,  step  unreliability 
was  then  compared  to  mean  unreliability  per  experimental  trial  studied. 
The  experimental  mean  value  was  '0.31935.  Assuming  a  trial  was  equivalent 
to >a  step,  the  ratio  of  the  ratio  of  the  two  numbers  gave  a  conversion 

I  1 

factor  to  multiply  times  the  experimental  unreliability. 

0.0026  =  0.008145 

0.31935 
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The  product  of  laboratory  unreliability  and  the  conversion  factor 
gave  an  unreliability  or  error  rate  for  a  dimension.  For  example, 
if  a  counter  1  inch  long  gave  a  lab  reliability  of  0.8773,  its 
unreliability  would  be  0.1227  (1-0.8773).  The  Data  Store  unreliability 
or  estimate  of  field  error  rate  for  that  dimension  would  be 

(0.1227)  x  (0.001845)  =  0.001 

giving  a  reliability  of  0.999.  Table  III  shows  a  portion  of  a  page 
from  the  Data  Store  Index  which  begins  with  this  reliability  for  the 
first  dimension. 


Table  III 

Counter  Dimensional  Reliability 
Rel i abi 1 i ty  Parameter  &  Dimension 


1. 

Size  (length) 

.9990 

a. 

1" 

.9998 

b. 

1-2" 

.9995 

c. 

3"  and  up 

2. 

Number  of  drums  (or  digits) 

.9997 

a. 

1-3 

.9993 

b. 

4-5 

.9985 

c. 

7  and  up 

3. 

Style 

.9995 

a. 

Continuously  rotating 

.9997 

b. 

Unit  jumps 

4. 

Use 

.9999 

a. 

Quantitative  read 

.9990 

b. 

Qualitative  read 

5. 

Numeral  legibility 

.9999 

a. 

Clear  and  concise 

.9994 

b. 

Potentially  ambiguous 

(From  Ref  15:4) 
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Another  element,  response  time,  listed  in  the  Data  Store,  is  not 
evaluated  in  this  study. 

The  purpose  of  the  data  was  to  evaluate  equipment  by  identifying 
design  features  which  would  degrade  operator  performance  and  providing 
guidance  concerning  selection  and  training  of  operators  for  evaluated 
equipment. 

Some  Limitations 

The  definition  of  error  will  be  covered  more  fully  when  the  results 
of  the  test  are  examined.  The  AIR  definition,  "seriously  degrade", 
seemed  to  be  an  individual  judgement  item.  The  conversion  factor  tied 
the  resultant  reliability  figures  to  general  field  reliability  and 
possibly,  totally  irrelevant  or  inadequate  laboratory  data.  Validation 
and  adjustment  of  the  data  was  made  on  a  narrow  group  of  equipment  with 
insufficient  reliability  data  to  make  any  statistical  analysis.  None¬ 
theless,  the  use  of  the  data  by  the  developers  of  other  methods 
confirmed  a  general  "in  the  ball  park"  validity.  Since  this  data 
still  remains  as  the  primary  bank  of  information,  a  continuation  of 
this  thesis  project  could  contribute  significantly  toward  human 
performance  evaluation. 

System  Development 

To  construct  a  system  which  would  validate  the  data,  an  apparatus 
was  constructed  with  two  panels  having  the  same  type  of  component 
displays  and  controls.  One  panel  contained  components  with  dimensions 
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as  closely  approximating  those  in  the  data  that  gave  the  highest 
reliability  and  the  other  with  dimensions  similar  to  those  having 
the  worst  reliability.  The  components  and  other  equipment  were  all 
off-the-shelf  or  experimental  equipment  not  currently  in  use.  Since 
some  of  the  equipment  was  experimental,  it  was  not  completely  pre¬ 
dictable  and  most  of  it  was  modified  to  perform  in  a  manner  not 
originally  intended.  The  panel  was  programmed  to  simulate  an  aircraft 
sequence  from  take-off  to  level-off  in  order  to  get  a  maximum  number 
of  logical  uses  of  the  components. 
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II.  EQUIPMENT  DESIGN 

The  system  had  two  main  groups  of  equipment.  The  display  and 
control  group  consisted  of  the  equipment  with  which  the  subjects  were 
associated.  It  included  two  panels  of  displays  and  controls  and  the 
platform  and  the  throttle  it  housed.  The  driving  equipment  included 
all  the  other  apparatus  for  operating  the  experiment  and  recording 
operator  responses.  More  technical  description  of  the  operation  of  the 
indicators  and  controls  is  included  in  the  equipment  section.  The 
panels  are  shown  in  Figures  1  and  2.  The  resulting  equipment  design 
was  a  function  of  individual  equipment  dimensions,  availability, 
adaptability  to  the  experiment,  possible  control,  and  panel  layout. 
Schematics  for  the  various  equipment  circuitry  is  included  in  the 
Appendix. 

) 

Indicators  and  Controls 

It  was  possible  to  use  five  types  of  indicators  on  each  panel: 
scales  that  were  semi-circular,  circular,  and  linear;  a  counter  and 
a  light.  The  controls  used  were  a  knob,  a  rotary  selector,  a  toggle 
switch,  a  push  button,  and  levers.  Sketches  were  drawn  of  the  best 
and  worst  case  for  the  parameters  in  the  Data  Store  Index.  From  these 
sketches,  the  search  began  for  the  indicator  and  control  components. 

It  v/as  obvious  that  many  parameters  and  categories  were  not  appropriate 
to  an  aircraft  instrument  application. 
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Panel  2,  Worst  Case 
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In  the  Data  Store  Index  there  are  an  average  of  nine  parameters 
for  each  indicator  and  control.  Generally  less  than  six  of  these 
parameters  could  be  used.  For  each  parameter  there  are  three  or  four 
values  for  each  parameter.  Quite  often  the  extreme  values  were  diffi¬ 
cult  to  use.  For  ex,  .pie,  a  semi-circular  scale  most  naturally  could 
be  used  for  aircraft  rate  of  climb  or  descent  (vertical  velocity).  The 
parameter,  scale  arc  length  (number  of  degrees  included  by  scope  face) 
has  the  categories  25°,  50-100°  and  200°.  The  lowest  category,1  25°, 
did  not  seem  to  allow  for  presentation  of  several  vertical  velocities 
and  graduations  between  them  so  only  two  categories  were  used.  The 
number  of  graduations  still  had  to  be  reduced  to  make  the  worst  case 
Indicator  legible.  For  the  parameter,  scale  interval  spacing  (distance 
between  graduation  marks),  the  reliability  figure  actually  turned  cut 
better  for  the  worst  case.  The  unly  useful  ammeter  that  could  be 
found  that  fit  the  50-100°  category  had  a  pointer  radius  in  the  same  i 
range  as  the  best  case  so  this  parameter  could  not  be  differentiated. 
Normal  aircraft  vertical  velocity  indicators  could  net  be  used 

their  variable  graduation  spacing  did  rot  lend  itself  to  the 

i 

index.  Similar  problems  were  encountered  differentiating  among  the 
'>the.  instruments.  New  faces  had  to  be  made  for  the  circular  engine 

I 

speed  indicators.  Again,  for  legibility,  the  small e»  instrument  had 

,  s 

to  have  fewer  graduations  and  a  better  reliability  for  that  parameter. 
The  vertical  scale  used  for  altitude  was  an  experimental  model. 

Two  identical  instruments  were  used.  To  create  a  worst  case,  the 
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presentation  area  was  reduced  and  the  number  of  graduations  was  increased. 
Counters  worked  out  best  for  airspeed.  A  fourth  digit  on  the  largest 
(best  case)  counter  was  taped  to  keep  the  reliability  at  least  equal 

i 

to  the  worst  case.  A  light  served  to  indicate  if  the  brakes  were  set. 

Levers  were  used  for  the  throttle  and  gear  handles.  A  push  button 
served  as  a  brake  button  and  a  rotary  selector  acted  as  a  flap  position 
control.  Another  type  of  notary  selector,1 a  knob,  was  wing  position 
control.  The  Index  used  the  knob  in  conjunction  with  a  separate 
display  but  for  simplicity  this  was  omitted.  A  three-position  toggle 
switch,  spring  loaded  to  the  center  position,  functioned  as  a  nose 
position  control.  .This  control  was  created  to  give  the  operator  a 

i  ' 

discrete  elevation  control.  The  normal  aircraft  control  wheel  or 

1  •  i 

stick  would  present  a  continuous  control  function  which  would  have 
been'  difficult  to  evaluate.  For  this  reason  and  experiment  complexity, 
a, heading  control  was  not  usfed. 

Placement  on  Panel- 

The  controls  and  displays  were  located  on  the  panels  in  the  most 

,  i 

functional  manner  possible.  The  panels  are  21  inches  wide  and  12  inches 
high  at  the  side,  with  the  top  curving  up  to  15  inches  in  height  at 

i 

the  center.  A  rcw  of  indicators  was  centered  on  a  horizontal  line 
10-1/2  inches  up  from  the  bottom  of  the  oanel.  This  located  them  just 
below  eye  level,  28  inches  from  the  eyes  of  an  average  male  subject 
Related  in  the  planned  positioh  at  the  cohsole.  Since  the  zero  position 
of  the  vertical  velocity  was  at  the  nine  o'clock  position,  it  was  placed 

13  . 
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on  the  right  side  of  the  panel.  To  balance  the  panel,  the  circular 
scale  was  put  on  the  left.  The  vertical  linear  scale  was  centered 
on  the  panel  to  use  the  maximum  panel  height  and  the  airspeed 
indicator  was  on  the  left  of  it.  Another  horizontal  line,  2  inches 
up  from  the  bottom  of  the  panel,  contained  most  of  the  controls  and 
the  brake  light.  This  placement  put  them  closer  to  the  hands  of  the 
subjects.  Since  both  the  knob  and  rotary  selector  were  similar  in 
operation,  they  were  placed  on  opposite  sides  of  the  panel.  The 
brake  light  and  button  were  symmetrically  below  the  altitude  indicator. 

To  balance  the  number  of  controls  and  manipulations,  the  toggle  switch 
was  placed  cn  the  right  side  of  the  panel.  The  gear  lever  was  placed 
to  the  extreme  left  of  the  panel  so  when  it  was  in  the  up  or  down 
position  it  would  not  shield  any  instruments  or  controls,  it  was 
also  above  the  line  of  other  controls  so  it  would  always  extend  well 
above  the  bottom  of  the  panel  and  :  .'.y  from  the  throttle,  which  was 
also  on  the  left  side.  One  throttle  was  used  with  both  panels.  It  is 
shown  in  Figure  3.  It  moved  fore  and  aft  in  a  control  box  which  was 
clamped  to  the  chair  of  the  subject.  It  was  placed  on  the  left  side 
to  locate  it  on  the  same  side  as  the  engine  speed  indicator  it 
controlled.  All  adjacent  controls  had  a  least  1-1/2  inch  clearance 
between  moving  surfaces.  All  controls  and  displays  were  labeled  in 
1/4  inch  letters  with  names  most  appropriate  to  what  they  showed  or 
did.  Several  labels  we^e  arbitrary.  Wing  position  was  not  the  generally 
used  term  for  wing  sweep,  yet  it  seemed  to  be  an  easier  understood 
description  for  the  actual  function  of  the  control. 
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Fig.  3 

Throttle  for  Both  Panels 


Driving  Equipment 

The  other  equipment  included  a  Tally  tape  reader,  tnree  Hunter 
IIIC  timers,  a  control  chassis,  a  white  noise  generator,  an  Esterline- 
Angus  recorder,  and  the  associated  wiring  and  electronic  equipment. 

The  tape  reader,  with  the  control  chassis  and  timers  on  top  of  it, 
is  shown  in  Figure  4.  Wiring  diagrams  for  the  equipment  are  included 
in  the  Appendix. 
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Fig.  4 

Tape  Reader,  Chassis, 
and  Timers 

All  the  panel  indicators  except  engine  speed  were  programmed  from 
the  Tally  reader.  The  Tally  read  eight  channels  of  punched  paper  tape 
(as  shown  in  Figure  4).  Within  the  Tally  console  were  two  relay  trees 
whose  outputs  controlled  relays  in  the  control  chassis.  These  control 
chassis  relays  operated  the  various  panel  instruments.  Both  panels 
were  driven  by  the  same  circuitry  except  for  the  airspeed  counters. 
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The  airspeed  of  Panel  2  (worst  case)  was  a  3-digit  impulse  counter 
driven  from  a  relaxation  oscillator  using  a  unijunction  transistor 
According  to  tape  code,  the  pulse  counter  could  be  stepped  at  two 
rates  or  stopped.  These  rates  could  be  adjusted  by  potentiometers  on 
the  chassis,  allowing  actuations  from  zero  to  20  pulses  per  second. 

This  allowed  airspeed  to  increase  well  beyond  normal  human  discrimina¬ 
tion  of  the  last  digit.  The  counter  had  a  push  button  reset  to  zero. 

The  other  airspeed  counter  was  geared  to  an  Electro-Craft  motor.  The 
motor  had  its  own  control  unit  which  had  a  relay  placed  inside  it  to 
allow  two-speed  control  from  the  control  chassis.  Two  additional 
potentiometers  also  permitted  individual  control  of  the  increase  in 
counter  reading.  Since  the  indication  on  the  counter  was  over  200  after 
the  run  and  it  had  to  be  reset  to  zero  for  the  next  run,  the  motor 
control  was  switched  to  the  reverse  position  and  the  Hunter  timer 
(which  is  labeled  Mo.  1  in  Figure  3  and  in  the  Appendix  schematic) 
was  connected  to  allow  runback  to  zero  for  the  selected  time  interval. 
The  speed  control  of  the  motor  was  used  to  allow  a  faster  reset  rate 
than  the  two  and  four  units-per-second  rates  set  in  the  control  chassis 
for  the  experimental  runs. 

The  moving  tape  of  the  altitude  indicators  was  controlled  from 
a  400-cycle  synchro  generator.  This  synchro,  on  the  control  chassis, 
was  positioned  in  steps  by  a  modified  stepping  switch.  The  switch 
stepped  one  position  every  time  an  altitude  code  appeared  on  the  punched 
programmer  tape.  The  reset  timer,  mentioned  earlier,  was  connected 
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■  i 

to  the  reset  coil  of  the  stepping  switch,  so  that  altitude  could  be 

reset  to  zero.  , 

) 

The  climb  meter  was  essentially  a  milliameter,  connected  through 

I 

one  of  three  rheostats  to  28  volts  d-c  or  disconnected  for  a  zero  rate. 
The  programming  tape  codes  selected  one  of  four  relays.,  which  in  turn 
selected  one  of  three  rheostats  for  the  three  climb  rates  or  no  rheostat 
for  zero  climb.  The  more  sensitive  meter  on  Panels  had  a  rheostat 
shunt  to  adjust  it  to  the  same  sensitivity  as  the  other  vertical 
velocity  indicator.  With  this,  arrangement,  the  three  rheostats  on  the 
control  chassis  were  used  to  adjust  the  indicator  on  Panel  1,  and  the 
indicator  on  Panel  2  could  then  be  corrected  with  its  own  rheostat  in 
back  of  the  panel.  1 

The  throttle  was  coupled  to  a  rotary  selector  switch  and  a  synchro 
generator.  The  selector  switch  routed  28  volt  d-c  to  the  appropriate 
channels  of  the  Esterline-Angus  recorder.  The  synchro  generator 'fed 

*  i 

throttle  position  information  to  a  synchro  repeater  of  the  engine 

1  i 

speed  indicator  on  the  panel. 

i  1 

Wing  position,  flap  position,  gear  and  nose  position  controls 
functioned  only  to  a  route  28  volts  to  the  various  channels  of  the 

»  i  1  ; 

Esterline-Angus  for  recording.  . 

1 

The  tape  reader  could  also  be  used' to  interrogate  various  control 

positions  during  a  program.  The  appropriate  code  resulted  in  :a  blip 

1 

on  the  Esterline  channel  corresponding  to  the  proper  position  of  thei 
throttle  or  flap  controls  if  the  control  was  not^  in  the  correct 

position.  No  mark  would  be  made  when  the  control  was  in  the  correct 

« 
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position  and  the  Ester-line  pen  properly  displaced.  This  feature  was 
not  used  in  the  experiment  because  during  the  fina"  debugging  period,, 

•  •  i:  1  i 

spurious  signals  actuated  instruments  erroneously  when  the  interrogation 

.  .  !  i 

codes  ran  through  the  reader.  ( 

The  speed 'at  which  the  Tally  read  the  signals  was  regulated  by  a 
pair  of  Hunter  timers|  (labeled  No.  2  and  No.  3).  The  sum  of  the  two 

,  i 

timer  settings  was  the  cycle  period.  The  most  convenient  cycle  period 
for  the-  tapes  was  one  second  so  each  timer  was  set  at  a  0.5-sec  interval. 

I  i 

These  timers  applied  pbwer  to  the  relay  trees  at  the  appropriate  time 

» 

of  the  read  cycle  and  moved  the  tape  through  the  reader.  1 


\ 


I 
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III.  THE  DESIGN  OF  THE  EXPERIMENT 


The  experiment  was  designed  to  obtain  a  maximum  number  of  output 
manipulations  by  four  trained  operators.  A  take-off  sequence  was 
selected  because  it  would  give  more  control  actuations  in  a  short  time 
and  appear  more  realistic  for  discrete  indications  and  a  pre-determined 
program.  By  using  a  low  altitude  level-off,  a  basic  checklist  was 
developed  with  17  realistically  spaced  control  actions  tied  to  various 
indicator  displays. 

Timing  was  developed  through  a  variety  of  runs  with  a  group  of 
trained  psychologists  and  subjects.  The  objective  was  to  condense  the 
sequence  into  tiie  shortest  time  in  which  a  trained  subject  could  read 
the  checklist,  observe  the  display,  and  perform  the  action  wihtout 
being  rushed  or  required  to  wait  too  long  for  the  next  indication.  The 
timing  turned  out  to  be  comparable  to  the  flight  sequence  of  a  high 
speed  aircraft. 


Flight  Profile 

The  flight  profile  used  is  shown  in  Figure  5.  The  base  (abscissa) 
is  time  in  seconds  from  brake  release.  The  ordinate  axis  is  marked 


in  units  of  airspeed  (knots)on  the  left  side  and  altitude  (feet)on  the 
right.  Airspeed  built  up  at  an  increasing  rate  for  the  first  25 
seconds,  then  more  slowly  as  an  aircraft  would  until  take-off  at  40 


seconds.  At  this  time  vertical  velocity  jumped  to  1000  feet  per  minute 
(FPM)  and  altitude  began  to  increase.  Airspeed  and  altitude  increased 
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Fig.  5 

Simulated  Flight  Profile 
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in  an  undulating  path  as  they  would  for  various  aircraft  power  and 
pitch  changes  until  level -off  at  3100  feet  and  approximately  212 
knots,  2  minutes  12  seconds  after  break  release. 

Varying  aircraft  performance  would  allow  for  many  realistic  flight 
paths.  For  variation  in  the  experiment,  two  alternate  sequences  were 
randomly  included.  One  started  out  with  a  higher  airspeed  and  lower 
ver' -cal  velocity  or  climb  rate  typical  of  an  aircraft  nose-low  altitude. 
The  other  sequence  simulated  a  nose-high  climb  with  lower  airspeed  and 
higher  vertical  velocity  initially.  Both  sequences  end  up  at  the  same 
standard  level-off. 

The  checklist  used  in  the  experimental  runs  is  shown  in  Table  IV. 

It  and  the  profile  were  carefully  arranged  to  provide  uniformly  spaced 
actions  throughout  the  run.  The  checklist  gave  an  action  to  be  performed 
on  a  control  for  various  indicator  displays.  Indicator-control  pairs 
were  varied  as  much  as  possible.  The  link  analysis  in  Figure  6  shows 
how  the  diversified  pairing  of  the  indicators  and  controls  is  used. 

The  average  indicator  and  control  use  was  3.1  times  per  run. 

The  students  used  the  checklist  during  the  runs.  The  first 
checklist  action,  pushing  the  brake  button,  started  the  sequence  on 
the  tape  reader.  Their  final  level -off  setting  left  the  controls  at 
the  initial  start  of  take-off  roll  configuration. 

Procedure 

Four  male  students  from  the  University  of  Dayton  were  used  for 
the  pilot  study  of  the  experiment.  The  coordinator  at  the  University 
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Table  IV 
Checklist 

The  actions  are  to  be  performed  after  obtaining  the  proper  indication. 
The  indications  will  always  appear.  Complete  each  action  in  the  numbered 
order,  even  though  the  succeeding  indication  might  appear  earlier.  Your 
objective  should  be  to  perform  the  actions  accurately  without  falling 
behind  the  sequence  of  indications. 

Indication  Action 


1. 

BRAKE  LIGHT 

BRAKE  BUTTON 

2. 

AIRSPEED  INCREASE 

- 

ENGINE  SPEED,  100  PER  CENT 

3. 

AIRSPEEC,  75  KNOTS 

- 

NOSE  UP 

4. 

AIRSPEED,  90  K 

- 

ENGINE  SPEED,  110% 

5. 

VERTICAL  VELOCITY, 

1000 

FPM 

- 

NOSE  UP 

6. 

ALTITUDE,  300  FT 

- 

GEAR  UP 

7. 

ALTITUDE,  550  FT 

- 

BRAKE  BUTTON 

8. 

VERTICAL  VELOCITY, 

2000 

FPM 

- 

FLAPS,  30  DEGREES 

9. 

AIRSPEED,  120  K 

- 

FLAPS,  01 

10. 

ALTITUDE,  1700  FT 

- 

ENGINE  SPEED,  100% 

11. 

AIRSPEED,  145  K 

- 

WING  POSITION,  40° 

12. 

ALTITUDE,  2550  FT 

- 

WING  POSITION,  60° 

13. 

ALTITUDE,  2800  FT 

- 

NOSE  DOWN 

14. 

VERTICAL  VELOCITY, 

1000 

FPM 

- 

ENGINE  SPEED,  80% 

15. 

AIRSPEED,  195  K 

- 

GEAR  DOWN 

16. 

ALTITUDE,  3100  FT 

- 

FLAPS,  60° 

17. 

VERTICAL  VELOCITY, 

0 

- 

WING  POSITIONS,  20° 
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of  Dayton  were  used  for  the  pilot  study  of  the  experiment.  The 
coordinator  at  the  University  was  asked  to  get  students  without 
flight  experience,  if  possible.  They  spent  three  hours  training  one 
week  on  one  panel  and  6  one-hour  sessions  testing  the  next  two  weeks. 

Two  subjects  tested  on  the  panel  on  which  they  had  trained,  two 
switched  to  the  other  panel.  The  third  week  all  four  students 
switched  panels. 

Table  V 
Panel  Usage 

. _ Panel _ 

Training'  Test. 

Subject  Week  1  Week  2  Week  3 

11  12 

2  1  2  1 

3  2  2  1 

4  2  12 

First  week  -  Two  subjects  at  a  time  were  taken  the  first  day  to 

avoid  duplication  and  increase  learning  by  observing  another  perform. 

They  were  given  a  short  instruction  sheet  (Appendix  B)  which  explained 
that  they  were  being  used  in  an  experiment  to  measure  human  performance. 
They  filled  out  a  questionnaire  (Appendix  C)  which  gave  scheduling  and 
location  information.  They  were  also  questioned  about  flight  or  tracking 
experience.  After  being  scheduled  for  the  remaining  activity,  they 
were  shown  the  two  panels.  Logical  relationships  and  analogies  between 
the  equipment  and  aircraft  indicators  and  controls  were  presented. 

The  subjects  were  encouraged  to  ask  questions  throughout  the  training. 
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The  subjects  than  sat  in  front  of  the  panel  and  went  through  the 
checklist,  manipulating  each  control  in  the  instructed  fashion  as  the 

i 

observer  explained  each  indication  that  would  be  observed.  It  was 
emphasized  that  each  action  had  to  be  performed  in’  order  before  looking 

I  I 

for  the  next  indication.  The  subjects  also  observed  the  othfer  parties 
being  instructed. 

The  observer  made  the  first  "live"  run  to  demonstrate  the  flow  of 
indications.  This  run  took  6  minutes  15  seconds  from  brake  release  1 
until  the  last  indication,  which  was  zero  vertical  velocity  (sequence 
21  or  S20,  Appendix  D).  The  run,  averaging  19  seconds  between  check-, 
list  indications,  had  three  20-30  second  periods  without  indicator 
changes  which  could  be  used  for  subject  questions.  The  tape  reader 
could  be  stopped  for  any  length  of  time,  if  necessary.  Each  subject 
then  performed  and  observed  this  run.  ' 

At  this  point  the  subjects  were  given  a  short  coffee  break. 

i 

While  they  chatted  they  were  handed  an  outline  of  the  panel  and  parts 
and  asked  to  name  the  indicators  and  controls.  Each  subject  then  returned 

I 

to  the  panel,  performed  and  observed  a  4  minute  10  second  run  and 
several  3  minute  10  second  runs.  The  average  time  between  required 
actions  had  been  reduced  down  to  15  and  12  seconds.  After  their 

'  j 

first  demonstration  run  the  subjects  wore  ear  phones  and'  heard  random 
noise  which  was  checked  before,  after,  and  intermittently  during  the 
runs.  The  random  noise  was  kept  well  below  any  uncomfortable  level 
yet  covered  a  frequency  range  and  volume  sufficient  to  block  out 
almost  all  outside  noise. 
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When  the  subjects  came  back  two  days  later  for  their  individual 

i 

training  hour,  they  performed  runs  until  they  had  completed  a  total  of 

six  3-minute  runs  and  three  2-minute  12-second  runs  similar  to  the 

■ 

actual  test  run. 

!  Second  week  -  At  the  first  session  of  the  second  week  each  subject 
was  given  each  one  of  the  three  tests.  After  completing  the  training, 
one  man. on  each  panel  switched  to  the  panel  on  which  he  had  not  trained. 
The  subjects  completed  as  many  runs  as  possible  the  first  day  and  45 
runs  for  the  week. 

I 

Third  week  -  All  four  subjects  switched  panels  for  the  last 

i 

45  runs  'this  week.  This  gave  each  subject  at  least  14  training  runs 

i 

on  dne  panel  and  45  test  runs  on  each  panel.  After  the  final  run 
was  completed  each  subject  was  questioned  briefly  about  his  impressions 
and  ideas  about  the  experiment  and  the  purpose  of  the  experiment  was 

I 

explained. 
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IV.  EQUIPMENT  OPERATION 

Within  the  limitations  of  laboratory  voltage  and  equipment 
operation,  every  effort  was  made  to  precisely  control  the  system.  This 
was  done  to  produce  uniform  subject  presentations  and  most  accurate 
possible  recording  of  their  responses. 

Laboratory  Power 

Unexpected  variation  in  the  line  voltage  to  the  equipment  hampered 
system  operation.  The  apparatus  was  set  up  on  the  third  floor  of  an 
Aeromedical  Laboratory  at  Wright-Patterson  Air  Force  Base.  This 
building  was  one  of  four  drawing  power  from  a  feeder  station.  In  all 
of  the  buildings,  variable  experimental  activity  was  conducted.  In 
the  basement  of  this  building  there  was  a  test  altitude  chamber.  When 
the  last  of  three  vacuum  pumps  for  the  chamber  was  activated,  a  voltage 
drop  of  several  volts  was  created  in  the  building.  According  to  power 
and  maintenance  personnel  for  the  area,  the  third  floor  of  this  building 
had  one  of  the  most  variable  voltages.  Intermittent  observations 
showed  a  variation  in  voltage  from  114-v  to  125-v  ac.  During  the  last 
day  of  runs  a  microcorder  tape  was  made  which  is  shown  in  Figure  7. 

The  instrument  was  dampened  and  did  not  show  large  temporary- 
voltage  changes.  Various  afternoon  times  are  shown  along  the  voltage 
line  which  is  indexed  at  115-v  and  125-v.  Arrows  show  unusual  breaks. 
Several  of  these  breaks  drop  to  115-volts  shortly  before  and  after 


28 


GRE/MATH/66-5 


CT H  B  .TP^'I  *v.—  *  t» .  — 

4%  .  f.  .-j*..*  ,  **  •«.  * 

}>'  :  I;.  :•  :>!  >  :»  {•  «;  <  o' 

3'i8 iV'V-i  i;  T:  »-&$<!>  )  i:.  L  ■ 
*  ‘1  -  -  °  -  j.  ij  -  •“* 

:■  i ;■  ii  r  i  .i.  i  i  :  t  i,  i  !i  j 


•f  ! 


-■3iJ 


■'"Tii  •’ 
:• 


•  •  il  I  . 

ll  * 

l?H 9 


S  ! 

-3-1-4  •  _• 


:0  .30  '  40  :50 


—  t4c6 


rn  1 


....  ,1 


-*1300 


-  iz$0 


_  1  .MIC  ZOCOfZDEfZ  '  . 

|  \~rkf£  - L / tJC.  CuTPuT  | 

Fig.  7 

Equipment  Line  Voltage  Input 


1300.  It  could  be  expected 
that  a  large  amount  of  equipment 
would  be  turned  on  right  after 
the  lunch  hour.  The  minimum 
consistent  voltage  was  reached 
in  mid-afternoon  and  built 
up  again  as  building  equipment 
was  turned  off  late  in  the  day. 

This  presented  a  problem  for 
exact  input  voltage  into  the 
system.  Sola  constant  voltage 
transformers  were  used  with 
the  most  important  equipment 
to  minimize  the  problem. 

Airspeed 

The  airspeed  on  Panel  1 
seemed  to  present  the  most  problems. 
The  counter  was  geared  to  an 


Electro-Craft  motor  unit  which  appeared  to  have  a  warm-up  effect  on 
airspeed.  The  final  test  run  sequences  had  approximately  the  same 
smount  of  run  time  and  were  programmed  to  end  up  at  the  same  end 
airspeed,  so  the  final  counter  reading  was  used  as  an  index  of  the 
operation  of  the  airspeec  circuitry.  It  was  found  that  this  final 
airspeed  would  gradually  build  and  peak  after  6-10  runs.  Airspeed 


GRE/MATH/66-5 


checks  with  precise  variable  outputs  did  show  that  the  airspeed  output 
was  also  a  function  of  line  voltage.  Isolating  other  factors,  a  drop 
from  1 28- v  to  116-v  produced  a  rise  in  final  counter  reading  of  10 
units.  Below  111 v  the  readings  dropped  again. 

About  the  time  the  test  runs  were  to  begin  it  was  discovered  that 
the  Hunter  timers  which  controlled  the  basic  system  pulse  were  also 
probably  heavy  contributors  to  the  variation.  Timar  variations  could 
have  been  related  to  the  times  between  runs,  times  between  actuations, 
the  actuation  period,  equipment  variation  (+ 5 t  for  117  +6  volts), 
and  input  voltages. 

A  test  run  sequence  took  2  minutes  12  seconds  with  approximately 
80  reconds  of  airspeed  motor  run  time.  Then,  1-2  minutes  of  reset 
time  which  included  around  11  seconds  of  fast  airspeed  runback.  Even 
if  a  warm-up  pattern  was  made  it  had  to  run  until  just  before  the 
subject  started  because  as  little  as  5  minutes  cooling  would  result 
in  the  same  increasing  output. 

The  final  hook-up  procedure  had  both  timers  and  the  airspeed 
control  plugged  into  the  Sola  transformer.  All  equipment  was  put  on 
warm-up  for  approximately  one  hour  before  subject  time.  Two  runs  were 
also  made  shortly  before  expected  subject  time  to  eliminate  an  initial 
low  outcome.  Figure  8  shows  bar  graphs  of  the  distribution  of  termina¬ 
tion  airspeeds  for  the  first  five  days.  Table  VI  shows  some  statistics 
of  these  distributions.  Included  is  range,  mean,  mode,  secondary  mode 
and  number  of  runs. 
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Table  VI 


Termination  Air  Speed  Statistics 


Day 

1 

2 

3 

4 

5 

.  6 

Lowest 

223.5 

213.5 

200.5 

211.5  , 

222 

203 

Highest 

233.5 

232.5 

225.5 

236 

23i.5 

222.5 

Mean 

228.8 

226.4, 

215.6 

226.3 

227  C  5 

214.3 

Mode 

229-231 

231-3 

209-211 

231-3 

225-227 

227-229 

— 

Secondary 

Mode 

225-227 

227-9 

219-221 

225-227 

227-229 

229-231 

- - 

No  of 

Runs 

25 

32 

33 

30 

30 

30 

Lowest 

210 

208 

i 

209 

209 

210 

210 

Highest 

212 

214 

215 

215 

215 

214*  ' 

Mean 

211.2 

212.0. 

214.3 

213.0 

213.5 

212.7 

214.2* 

5 

Mode 

211 

213 

214 

213 

l 

■  214 

213 

Panel 

214 

213 

2 

Secondary 

Mode 

212 

214 

213  » 

— 

215 

i 

214 

No  of 

18 

33 

39 

30 

30 

30 

Runs 

i 

*214.2 

average 

i 

includes 

3  runs 

ending  at 

228 

I 
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i  , 

Several  alternate  hook-ups  were  tried  the  last  day  and  a  variac 
was  used  -y  itself  to  test  control  and  also  attempt  to  bring  the 

'  I  i  - 

average  run  times  of  all  subjects  closer  together.  Interestihgly 

J  ’  4 

enough,  although  the  range  of  end  speeds  ,(as  indicators,  generally,  of 

•  '  i 

speed  relationships  to  other  instruments  throughout  the  run)  was 
'  larger  than  desired,  the  average  end  speed  seen  by  all  four  subjects 

,  I  t  t  '  ' 

varied  by  only  three  units.  This  means  that  the  average  time  at  which 

I  ,  ■  ' 

■  t 

a  subject  saw  a  specific  airspeed  during  the  runs  varied  less  than  2 

seconds.  •: This 'would  indicate  that  for  a  larger  number  of  runs,  if 

each  subject  ran  at  different  times  of  the  day,  ^he  distributions  fbr 

all  subjects  would  probably  be  equivalent. 

,  ‘  1  !  ;  '  '  , 

1  It  would  be  suggested  for  future  experiments  that  the  variac 

be  hooked  in  the  line  after  the  Sola  transformer.  Because  o,f  the  low 

> 

•  i  * 

output  voltage  of  the  Sola,  this  would  give  some  variable  control 

i  ^  i 

.over  airspeed-  To  eliminate  the  warm-up  range  of  output,  several 

1  ;  i 

sequences  could  be  run1  just  before  the  subject  started  (with  the  400 

■  '  ,  1  ■ 

.cycle  power  turned  off  to  'cut  out  altimeter  actuation),  if  this  could 

not  bq  done,  the  variac  could  be  set  low  and  gradually  turned  up  as 
the  system  warmed  up.  Runs  could  also  be  scheduled  to  avoid  general 

i 

building  turn-on  and  turn-off  activity.  • 

The  airspeed,  of  Panel  1  shows  less  effect  of  voltage  variation 
and  would  require  little  attention.  One  unexplanabTe  variation 
should  be  avoided.  When  the  timers  were  plugged  into  the  variac  (which 
only  allows  some  control,  not  constant  voltage)  the  final  counter 
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readings  jumped  to  228,  a  rise  of  approximately  15  units.  No  variation 
of  the  variac  could  regulate  this  output  so  they  had  to  be  plugged 
into  the  line  voltage  or  the  Sola  transforme*'. 

Altimeter 

The  altimeters  were  experimental  instruments  and  no  wiring  diagrams 
could  be  found  on-base,  with  the  distributor,  or  at  the  factory.  Guess 
work  was  involved  determining  the  wiring,  so  complete  control  was 
sometimes  questionable.  The  synchros  in  the  instrument  were  geared 
to  drive  the  movable  tape  through  its  entire  11,500  foot  range.  The 
fine  synchro  made  approximately  17  revolutions  to  one  revolution  of 
the  gross  synchro  (schematic  in  Appendix). 

Originally  the  altitude  was  programmed  to  go  to  5000  feet.  When 
the  reset  timer  was  actuated,  the  step  switch  driving  the  synchro 
generator  would  spring  back  to  the  zero  position.  The  instrument 
gross  synchro  had  been  switched  on  and  attempted  to  follow  back  to  the 
nearest  zero  position.  At  the  completion  of  the  reset  timing,  the 
fine  synchro  was  re-activated  and  returned  to  the  nearest  zero 
position.  With  repeated  use,  the  gross  synchro  seemed  to  return 
to  large  negative  values  or  to  a  position  that  upset  the  fine  synchro 
when  it  took  over  an  it  would  not  go  completely  to  its  original  zero 
point  or  would  go  beyond  to  one  complete  revolution  and  indicate  6500 
feet.  This  condition  could  not  be  resolved  in  the  short  time  before 
subjects  were  to  be  run  so  the  gross  synchro  was  disconnected  and 
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the  program  sequence  was  re-written  to  keep  the  fine  synchro  in  a 
range  (less  than  180  degrees  of  movement  or  3300  feet  of  aUitude) 
that  would  allow  it  to  return  to  the  zero  point  after  each  run.  There 
remained  some  variation  in  the  altitude  indications  but  the  checklist 
was  organized  to  avoid  ambiguous  readings.  The  subjects  were  required 
to  note  a  specified  altitude  plus  or  minus  50  feet  before  making 
control  actuations. 

The  stepping  switch  increased  altitude  approximately  280  feet  in 
each  incremental  step.  The  variation  in  the  instrument  seemed  to  be 
caused  by  a  spring  in  the  instrument  which  opposed  the  movement  and/or 
an  uneven  torque  sometimes  caused  by  a  slight  shift  in  phase  of  the 
400-cycle  input  voltage. 
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V.  TEST  RESULTS  AND  ANALYSIS 


The  experiment  was  designed  as  a  pilot  study  for  further  system 
testing.  It  was  important  to  define  and  analyze  errors,  not  only  to 
determine  reliability  but  to  design  the  equipment  and  the  test.  The 
following  discussion  might  aid  later,  more  sophisticated  testing. 

Error  Definition 

The  basic  error  definition  in  AIR  material,  an  action  that  would 
degrade  the  system,  seemed  too  narrow.  Of  the  many  definitions,  one 
of  the  more  acceptable  (Ref  16  and  Ref  15)  was: 

1.  Perform  required  action  incorrectly 

2.  Perform  required  action  out  of  sequence 

3.  Perform  non- required  action 

4.  Failure  to  perform  required  ;acti on. 

An  interview  with  Mrs.  Sarah  Hunger,  AIR  research  scientist, 

(Ref  26)  convinced  the  author  that  "near"  errors  could  have  significant 
effects,  give  more  data,  and  should  be  recorded.  So,  any  incorrect 
response  (with  some  modification)  was  recorded  and  classified  as 
either  actuation  or  respon:°  error.  An  actuation  error  was  a  definite 
indicator  misreading  or  incorrect  control  movement  by  the  subject 
(slightly  modified  by  later  discussion).  A  "near"  or  response  error 
was  a  start  toward  the  wrong  control.  With  this  consideration  and  for 


a  back-up  of  the  recording  system,  all  subject  runs  were  observed. 


GRE/MATH/66-5 


An  observer  sheet  was  developed  which  reduced  the  distraction  from 
subject  operation  of  the  panel. 

Observation  Sheet 

The  observation  sheet  (see  Appendix  A)  was  a  8  x  10-1/2  inch  General 
Purpose  Data  Sheet,  with  extra  columns  added.  The  correct  action  and 
indication  for  each  step  are  on  the  left  side  of  the  sheet.  As  an  aid 
to  the  observer,  the  action  he  is  watching  is  also  repeated  on  the 
right  side  of  the  sheet.  Two  columns  for  each  of  the  five  controls  and 
a  comments  column  spread  from  left  to  right.  One  of  the  two  columns  is 
marked  A  for  actuation  errors,  the  other  column  is  marked  R  for 
response  error.  The  observer  checks  the  block  for  the  incorrect 
control  action  made  (response  or  actuation)  on  the  correct  response- 
indication  line.  Display  errors  could  be  recorded  by  circling  the 
indication. 

A  sheet  was  used  for  each  run.  The  starting  time  and  ending 
airspeed  counter  readings  were  recorded  for  equipment  evaluation. 

After  the  testing,  the  checked  blocks  were  compared  with  the  Esterline- 
Angus  recording  and  tabulations  made  for  the  various  component  errors. 

Error  Amplification 

Further  amplification  of  errors  was  necessary  for  system  components. 
The  subject  sometimes  moved  the  throttle  through  the  correct  position. 

If  he  immediately  moved  it  back  to  the  correct  position,  it  was  not 
counted  as  an  error.  The  average  subject  took  about  1-1/2  to  2  seconds 
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to  glance  from  the  throttle  or  any  place  on  the  panel,  observe  an 
Incorrect  engine  speed  and  change  it.  If  he  appeared  to  do  more  than 
this  action  or  take  longer  (depending  on  the  movement  rate  of  the 
subject)  he  was  credited  with  an  error.  Movement  in  the  wrong  direction 
was  considered  an  error. 

For  both  actuation  and  response  errors  it  was  helpful  to  understand 
the  mannerisms  of  the  subjects.  The  movement  rate  for  subjects  varied. 
One  man  normally  took  longer  to  glance  around  or  move  a  dial  and  if 
the  1-1/2  to  2  second  rule  was  rigidly  maintained,  some  of  his  normal 
rate  actions  would  have  been  considered  as  actuation  errors.  Conversely, 
a  faster  acting  man  might  be  committing  a  response  error  if  he  left  the 
control  incorrectly  positioned  longer  than  he  normally  would  take  to 
observe  the  incorrect  indication. 

One  subject  always  placed  his  hand  on  the  next  control  to  be 
actuated  long  before  the  necessary  indication.  Observing  one  of  the 
other  subjects,  the  occasional  time  he  reached  out  to  manipulate  a 
control,  it  appeared  he  was  making  a  response  error. 

Other  Error  Effects 

Unfortunately,  many  errors  appeared  which  could  not  be  specifically 
attributed  to  indicator  or  control  characteristics  nor  eliminated  by 
experiment  design.  A  checklist  was  used  because  of  the  number  of 
experiment  steps  and  allowable  training.  Twice  a  subject  omitted  a 
step.  These  omissions  might  be  attributed  more  properly  to  the  checklist 
or  the  pacing  of  events  in  the  sequence.  The  checklist  was  constructed 
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to  give  a  logical  progression  of  events  but  some  did  not  appear  as 
realistic  as  desired.  One  uncomfortable  action  for  the  subjects 
required  putting  the  Nose  Position  up  when  the  vertical  velocity  read 
1000  feet  per  minute  (FPM).  One  subject  said  he  always  wanted  to  do 
something  else  at  altitude  1700  feet,  rather  than  pull  power  back  to 
100%. 

The  proximity  of  one  control  to  another  was  a  significant  factor, 
especially  with  the  gear  and  throttle  levers.  It  was  intended  to  have 
the  subject  seated  with  his  eyes  25-28  inches  from  the  panel,  hori¬ 
zontally  in  line  with  the  top  row  of  indicators.  This  arrangement 
located  the  edge  of  the  throttle  12  inches  from  the  gear  lever  at 
the  closest  position.  Spacing  between  the  wall  and  the  chair  in  the 
stall  in  which  the  subject  worked  was  limited  and  he  consequently 
tended  to  move  forward  to  a  position  6  inches  closer.  This  reduced 
the  eye-to-panel  distance  and  throttle-gear  separation.  This  gear- 
throttle  proximity  error  seemed  a  more  important  factor  than  their 
dimensions.  This  proximity  error  showed  up  for  all  controls.  The  flap 
and  wing  sweep  were  placed  on  opposite  sides  of  the  panel  to  minimize 
errors  due  to  their  similarity  of  shape  and  movement  but  this  also  was 
significant.  Several  times  an  improper  control  movement  at  one  point 
caused  subject  confusion  with  possible  response  and  actuation  errors 
when  the  next  use  of  that  control  was  required. 

Recording  and  Observer  Errors 

Also  present  were  equipment  and  observer  errors.  The  observer 
monitored  equipment  and  watched  subjects  for  over  four  hours  every  day. 
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trying  to  record  and  interpret  their  errors,  while  also  conside’ ing 
equipment  operation.  He  also  reset  and  tested  the  equipment  and' 
recorded  its  operation.  No  doubt  fatigue  played  a  role,  especially 
later  in  the  day.  To  complete  runs  as  fast  as  possible,  the  subjects 
were  started  on  succeeding  runs  as  soon  as  the  observer  could  get  back 
to  his  seat.  There  were  slight  observer  distractions  as  the  runs 
started  and  these  and  the  time  delay  for  close  concentration  might 
have  caused  him  to  miss  errors  early  in  the  sequence.  He  also  might 

<  j  > 

have  missed  indicators  of  proper  control  movement.  These  we?'.* 
as  clicks  into  detents  or  the  absence  of  clicks  as  controls  were 
actuated  properly.  All  these  little  possibilities  became  signi. 
when  unexpected  Esterline-Angus  recording  errors  and  its  reduced 
reliability  appeared. 

i 

On  the  last  six  runs  of  the  testing,  when  the  gear  was  to  be 
lowered  at  a  195  knot  airspeed,  the  recorder  showed  in  error.  It 
indicated  that  the  gear  was  taken  out  of  the  up  position  but  never 
went  to  the  down  position.  The  subject  was  not  rushed  on  these 
particular  runs,  in  a  situation  when  he  might  have  slapped  at  the 

I 

control  and  would  not  have  completely  activated  the  control.  The 
observer  was  sure  that  the  subject  had  completely  moved  the  gear 
lever  in  most  of  these  cases  and  was  not  developing  an  error  habit. 

I 

Also  the  Esterline-Angus  several  other  times  exhibited  what  was 
determined  to  be  group  erroneous  signals.  For  five,  almost  consecutive 
runs,  the  recorder  showed  no  blip  at  Wing  Position  60°  when  required. 
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thus  'indicating  a  position  between  the  40°  and  60°  detents.  This 
should  have  meant  that  the  subject  left  the  selector  at  50°.  On  the 
next  step  when  the  subject  Drought  the  control  to  20°,  the  recorder 
showed  a  blip  at  60°,  as  though  the  control  had  gone  through  that 

i 

position.  Since  60°  was  a  limit  position  and  the  subject  would  not 
•go  through  it  going  to  20°,  for  this  series  of  runs  it  was  judged 

I 

there  was  no  subject  error.  Efforts  to  duplicate  this  situation  and 
Esterline-Angus  error  indications  were  unsuccessful . 

The  tabulation  of  errors  is  shown  in  Table  VII.  It  i.  bn  en 
down  by.  day  for  each  pandl  and  subject.  Each  block  of  the  action 

shows  the  actuation  plus  response  error  totals  for  each  panei,  subject, 
day»  and  test.  Table  VIII  gives  the  corresponding  number  of  runs 
made  for  each  subject  and.  day.  Subject  tardiness,  equipment  problems 

i  • 

and  the  12  training  runs  given  the  first  day  prevented  an  equal  dis¬ 
tribution  of  runs  the  first  week  and  any  comparison  of  daily  errors 
should  consider  this. 

Subjects  2  and  3  trained  on  Panel  1.  For  the  first  week  subjects 

! 

3  and  4  ran  Panel  1.  They  rotated  panels  the  second  week.  There 

i 

were  12  training  runs  the  first  test  day  on  the  panels  on  which  the 
subjects  were  to  begin  the  test.  For  subjects  2  and  4,  these  12  runs 
were  their  only  runs  on  the,  particular  test  panel,  before  the  test 
began.  Subjects  1  and  3  got  to  train  and  practice  on  the  panels  they 
began  the  test: 

The  results  did  show  fewer  errors  on  Panel  t,  substantiating  the 
general  design  preference,  although  this  was  hardly  adequate  validation 

I 
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Table  VII 


Test  Errors 


Day 

1 

2 

3 

4 

5 

6 

Panel 

1 

2+1 

0 

1+3 

2+6 

0+6 

0+9 

5+25 

2 

0+8 

4+12 

6+10 

2+0 

2+1 

0+2 

14+33 

S 

U 

1 

0+5 

2+3 

4+6 

1+5 

0+4 

0+7 

7+30 

B 

J 

2 

0+3 

2+9 

2+4 

1+1 

0+2 

0+2 

5+21 

E 

C 

3 

2+0 

0 

1+2 

2+0 

1+1 

0+2 

6+5 

T 

4 

0+1 

0 

t 

0+1 

0 

1+0 

0 

1+2 

♦First  number  is  Actuation  Error 
Second  number  is  Response  Error 
2+1  means  two  Actuation  Errors 
and  one  Response  Error 


Table  VIII 


Test  Runs 


Day 

1 

2 

3 

4 

5 

6 

Totals 

Panel 

1 

25 

32 

33 

30 

30 

30 

180 

2 

18 

33 

39 

30 

30 

30 

180 

S 

U 

1 

9 

16 

20 

15 

15 

15 

90 

B 

J 

2 

9 

17 

19 

15 

15 

15 

90 

E 

C 

3 

10 

17 

18 

15 

15 

15 

90 

T 

4 

15 

15 

15 

15 

15 

15 

90 

Runs  Totals 

43 

65 

72 

60 

60 

60 

360 

T* - ‘r 


GRE/MATH/66-5 


Total  errors  did  decrease  the  second  week  and  observations  of  the 
subjects  and  interviews  with  them  later  confirmed  that  learning  was 
still  taking  place.  Unexpectedly, the  best  performer  had  to  shift 
panels  for  testing  and  the  worst  performer  remained  on  the  same  panel 
he  had  trained. 

Effect  of  Non-Subject  Errors 

Unfortunately  malfunctions  of  control,  display  and  recording 
equipment  were  numerous  enough  to  seriously  degrade  test  results  if 
there  was  not  considerable  redundancy  in  error  observation.  This 
probability  led  to  the  early  decision  to  observe  every  run,  manually 
record  errors,  spot-check  and  monitor  equipment,  and  carefully  interpret 
Esterli.-.e  recorder  readings.  This  error  analysis  resulted  in  an  error 
tree  (Figure  9,  for  Panel  1). 

To  be  counted  as  an  error,  a  correct  action  had  to  be  incorrectly 
recorded,  interpreted,  and  observed;  or,  an  error  had  to  be  correctly 
recorded,  interpreted,  and  observed.  Of  course  there  was  a  great  deal 
of  interplay,  especially  between  interpretation  and  observer  detection 
but  simply,  line  RsQrQiQo  represented  incorrectly  determined  errors, 
line  QsRrRiRo  represented  properly  determined  errors.  Analysis  of  the 
recorder  before  and  during  the  tests  indicated  it  had  an  accuracy  of 
0.999  (20  errors  in  more  than  18,000  movements).  Psychology  literature 
continually  emphasized  observer  accuracy  of  0.85  (although  I  dispute 
this  low  figure)  and  events  during  use  of  the  Esterline-Angus  pointed 
to  an  interpretation  accuracy  of  0.95.  Using  these  numbers  and  analysis, 
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the  20  indicated  errors  on  the  recorder  were  reduced  to  one  actual 
recorder  error  which  matched  the  one  error  missed  by  all  the  redundancy. 
This  is  approximate  logic,  used  only  to  verify  the  magnitude  of  error 
determination,  not  substantiate  exact  numbers. 

Panel  Analysis 

Table  IX  shows  a  conversion  of  reliabilities  (from  the  Data  Store 
Index)  to  errors  for  1 0,000  uses.  For  each  step  of  the  checklist,  the 
indicator  and  control  pair  is  listed  plus  the  errors  for  10,000  uses 
and  a  resultant  reliability  for  the  pair.  This  reliability  includes  an 
identification-discrimination  effect  which  is  approximately  two  errors 
per  10,000  uses.  Interestingly  enough,  the  product  of  these  three 
high  reliability  numoers  is  closely  related  to  the  sum  of  errors 
per  10,000  uses.  The  product  is  almost  equal  to  the  ratio  of  10,000 
minus  the  sum,  over  10,000. 

As  the  AIR  Data  Store  indicates,  most  errors  could  be  expected 
fro.,  the  gear  and  throttle  levers;  the  experimental  results  agreed. 

If  the  data  for  a  lever  (as  it  does  for  some  components)  contained  a 
parameter  for  proximity  and/or  similarity  of  controls  it  would  be 
possible  to  evaluate  this  effect. 

According  to  the  Data  Store,  the  most  numerous  display  errors 
could  be  attributed  to  the  semi-circular  scale,  vertical  velocity. 

The  qualitative  manner  in  which  this  was  used  (asking  the  subject  to 
read  whole  1000's  of  feet  within  +200  feet)  seemed  to  reduce  this 
error. 
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'  ahU  IX 


Panel  Analysis  -  AIR  Data 

Panel  1  Panel  2 

•nJica tor-Control  Pair  Errors  .  Reliability  Errors  Reliability 


1 

Light  &  Brake  Button 

9+8* 

.9981 

13+10  = 

.9975 

2 

Airspeed  &  Throttle  Lever 

82+14  = 

.9902 

82+20  =• 

.9896 

3 

Airspeed  &  Nose  Position  . 

15+14  = 

.9969  • 

17+20  =i 

.9961 

t  1 

4 

Airspeed  &  Throttle  Lever 

82+14  = 

.9902 

82+20  = 

.9896 

5 

Vertical  Velocity  &  Nose 
Position 

15+149  = 

.9834  ■ 

17+152  « 

»  } 

.9829 

6 

Altitude  &  Gear  Lever 

174+58  :  = 

.9766  * 

179+86  = 

.9734 

7 

Altitude  &  Brake  Button  . 

9+58 

.9931 

13+86 

.9899 

8 

Vertical  Velocity  & 

Flap  Control 

21+145  = 

,9828 

44+154  = 

.9802 

9 

Airspeed  &  Flap  Contrpl 

21+13  = 

.9964 

44+20  r 

.9936 

10 

Altitude  &  Throttle  Lever 

82+58  -- 

.9859 

:  82+86  = 

.9831 

11 

Airspeed  8  Wing  Sweep 

14+14  = 

.9970, 

19+20  = 

1 

.9959 

12 

Altitude  &  Wing  Sweep 

14+58  1  = 

..9926 

19+8,6  = 

,9893 

13 

Altitude  &  Nose  Position 

15+58  = 

.9925 

17+86  * 

.9895 

14 

Vertical  Velocity  8 
Throttle  Lever 

82+149  = 

.9768 

82+152  = 

.9765 

15 

Airspeed  &  Gear  Lever 

174+14  - 

.9810 

179+20  = 

.9799 

If 

Altitude  R  Flap  Control 

21+58  = 

.9919 

44+86  = 

.9868 

17 

Vertical  Velocity  & 

Wing  Sweep 

14+149'  = 

.9835 

19+152'  = 

.9827 

Panel  Reliability 

.8251 

.7970 

♦The  numbers  represent  the  exDected  errors  for  10,000 
uses  for  each  component.  Hence,  9  errors  could  be 
expected  using  the  light,  8  for  the  brake  button  and 
2  for  discrimination,  yielding  a  combined  reliability 
of  0.9S81 . 
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i 


A  table  of  actuation  error  rates  for  the  system  components  is 

l 

shown.  Components  are  listed  in  the  center  and  Data  Store  prediction 
and  experimental  results  for  actuation  errors  are  listed.  All  actuation 

i  1  i 

errors  with  Panel  1  were  with  the  throttle.  For  Panel  2,  errors  were 

j  i 

made  with  all  the  controls,  except  the  brake  button.  Since  the  air¬ 
speed  was  a  component  of  throttle,  i,t  was  not  evaluated  per*  se. 

Probably  a  more  accurate  analysis  would  giye  more  weight  to  response 

■  i  <  • 

errors  and  consequently  a  closer  approximation  to  the  Data  Store 
valiies.  i 

I 

Table  X 

Comparison  of  Data  Store  and 
System  Component  Reliability 


I 


>  Panel  1 

Component 

Panel  2 

l 

Data  Store 

Experiment 

Data  Store  Experiment 

.9986 

1 .0000 

Counter 

.9980, 

1 .0000 

.9992 

1.0000 

Light 

.9990 

1 .0000 

.9942 

1.00Q0 

Linear  Scale 

.9914 

1.0000 

^9851 

1.0000 

1 

Semi-circular 

.9848 

i.ooop 

.9986  . 

1.0000 

Knob 

.9981 

.9926 

.9918 

.9907 

Throttle 

.9918 

.9907 

.9826  . 

1.00C0 

1 

Gear 

.9821 

.9926 

,  .9991 

1.0000 

Brake  Button 

.9987 

1 .0000 

.9979 

1 ^  0000 

Rotary  Selector. 

'  .9556 

.9975 

,$985 

1.0000  , 

Toggle  Switch 

.9983 

.9926 
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Subject  Debriefing 

The  subjects  were  questioned  after  the  experiment.  They  agreed 
that  the  throttle  and  gear  proximity  and  wing  position  and  flap 
control  similarity  were  prime  error  effectors.  The  altimeter  was 
the  most  difficult  to  read  yet  its  indication  jumps  drew  their 
attention.  They  felt  the  training  was  adequate. 

When  asked  about  the  specific  purpose  of  the  panel,  most  felt 
it  was  to  test  the  equipment,  panel  design  or  readability.  One 
man  gussed  that  the  purpose  was  to  determine  subject  response  to 
different  controls,  timing,  and  errors. 
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VI.  CONCLUSION 


A  system  was  designed,  built  and  operated  to  validate  human 
reliability  data.  A  great  number  of  variables  appeared  in  the  design 
and  operation  of  the  equipment.  These  variables  were  eliminated, 
avoided,  controlled  or  reduced  to  the  point  they  had  minimal  effect 
on  the  evaluation  of  subject  performance.  Equipment  operating 
instructions  in  Appendix  B  would  maintain  the  best  equipment  per¬ 
formance  obtained  during  the  study.  The  experimental  results  would 
be  imporved  considerably  in  an  area  with  more  constant  voltage  and 
with  the  recommended  equipment  improvements. 

The  subject  comments  and  their  performance  indicate  that  the 
pacing  and  timing  evolved  for  the  runs  were  good.  Their  overall 
performance  improved  only  slightly  during  the  testing  and  their 
comments  indicate  that  the  pre-test  training  was  adequate 

The  redundancy  used  to  determine  actual  errors  proved  to  be 
the  most  valuable  foresight.  Further  studies  might  film  the  subject 
from  an  angle  which  would  allow  subsequent  recheck  of  his  movements. 
This  would  eliminate  observer  error  and  allow  for  permanent  records 
which  could  be  analyzed  in  any  desirable  manner.  This  would  be 
particularly  fruitful  for  problems  that  occur  after  testing  had 
begun  and  the  procedure  could  not  be  altered. 

The  relative  validity  of  the  AIR  Data  Store  Index  and  some  of 
the  instruments,  indicators  and  combinations  were  confirmed  for  the 
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two  panels.  The  system  could  serve  as  a  broad  base  for  further 
research  on  the  parameters  of  individual  indicators  and  controls. 
Simple  tasks  could  be  performed  by  masking  or  removing  many  of  the 
indicators  or  controls.  This  would  allow  a  larger  number  of  short 
runs  for  statistical  analysis  of  individual  units  and  combinations. 
Using  the  panels  as  a  breadboard,  variations  in  dimensions  and 
instruments  could  be  studies.  Pacing  could  be  varied  and  the  reaction 
times  correlated. 

Population  averages  and  variances  of  error  could  be  obtained. 

With  the  data  collected,  different  mathematical  models  could 
be  derived  to  predict  reliability.  Basic  human  error  rate  data 
could  be  generated. 

The  variables  of  equipment  operation  could  be  carefully  analyzed 
and  the  test  equipment  and  results  improved.  The  various  error 
effects  would  be  a  fruitful  study  area  to  improve  methods  and 
validation.  Particularly  useful  might  be  the  response  errors  as 
defined  and  measured  in  this  study. 

Quantification  of  human  reliability  data  is  a  young  and  fertile 
field.  Hopefully,  this  study  has  added  to  the  enrichment  of  that 
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Appendix  A 

Observation  Sheet 


GENERAL  PURPOSE  DATA  SHEET  (t  >  10h) 
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i  1  Appendix  B  , 

Instructions 

You  are  being  used  in  an  experiment  to  measure  human  performance. 
Following  a  checklist,  you  will  observe  indicators  and  operate  controls 
bn  two  panels  which  resemble  an  airplace  cockpit  panel.  These 

i  i 

panels  each  have  the  same  locations.  The  dimensions  and  other  para¬ 
meters  of  these ■ indicators  and  controls  vary  for  the  panels. 

i  You  will  be  trained  to  operate  a  panel  in  a  manner  similar 
to  an  aircraft  sequence  from  take-off  to  le\el-off.  First  you  will 

I  » 

be  given  a  brief  description  of  the  equipment  and  the  general  relation¬ 
ships  to  actual  aircraft  equipment.  The  checklist  will  then  be 
explained  and  you  will  operate  the  controls.  The  observer  will  make  a 
slow  demonstration  run  and  you  will  perform  the  same  run.  After 
training, and  practice  runs,  you  will  ‘perform  experimental  runs. 

1  This  first  period  will  last  two  hours,  (with  a  short  break)  and 
the  succeeding  five  periods,  one  hour  each.  Questions  will  be 

answered  anytime  during  your  training.  Please  complete  the  questionnaire 

\ 

and  arrange  for  the  next  appointment  before  proceeding  to  the  experimental 
booth. 

Fig.  11 

Subject  Instructions 
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Appendix  C 
General  Information 


NAME: _ 

AGE: _ 

PHONE  NO:  _ 

BEST  TIMES  TO  CALL:  _ 

OTHER  TIME  DEFINITELY  AVAILABLE:  _ 

WILL  YOU  NEED  TRANSPORTATION?  _ 

HAVE  YOU  PARTICIPATED  IN  ANY  TRACKING  STUDIES? 

HOW  MANY  HOURS  (APPROX.)?  _ 

WHAr  FLIGHT  EXPERIENCE  DO  YOU  HAVE? 


Fig.  12 

Sject  General  Information 
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Appendix  D 

Typical  Tape  Sequence 


Table  XI 

Sequence  24(S24) 


Time 

Relay 

Checklist 

(Sec) 

Action 

A/S 

Action 

Relay  Action  gives  indicator  being 

activated. 

A/S,  Airspeed  .S/0,  S/1, 

1 

S/1 

Brake  Button 

S/2  are  the  airspeed  motor  speeds 

2 

S/0 

2 

of  0 

,  2  KT/Sec,  4  KT/Sec.  V/0, 

3 

S/2 

100% 

V/1, 

V/2  are  vertical 

velocity 

19 

S/0 

66 

indications 

of  0,  1000  FPM,  2000 

20 

S/2 

Nose  Up 

FPM. 

A-l, 

A-2,  .  .  . 

are  al timeter 

24 

S/0 

82 

actuations 

in  increments  of  280 

25 

S/1 

ft. 

Checklist  Action 

1  shows  required 

27 

S/0 

86 

action  by  operator. 

28 

S/1 

100% 

32 

S/0 

94 

94 

S/0 

150 

34 

s/1 

96 

S/I 

37 

S/0 

100 

97 

A-8 

39 

s/1 

98 

S/0 

154 

42 

S/0 

106 

99 

S/2 

44 

v/1 

Nose  Up 

103 

S/0 

170 

45 

s/1 

104 

A-9 

Wing  Sweep  60° 

47 

S/0 

no 

105 

S/1 

49 

s/1 

107 

S/0 

174 

50 

S/0 

112 

109 

S/2 

54 

A-l 

Gear  Up 

no 

S/0 

178 

Nose  Down 

57 

s/1 

111 

A-10 

58 

S/0 

114 

112 

S/2 

62 

A-2 

Brake  Button 

113 

V/2 

66 

V/2 

Flaps  30° 

115 

S/0 

190 

68 

S/1 

117 

V/1 

80% 

71 

A- 3 

Flaps  0 

118 

S/1 

73 

S/0 

124 

121 

S/0 

196 

Gear  Dow.) 

76 

A- 4 

100% 

123 

S/1 

78 

S/0 

130 

128 

S/0 

206 

79 

S/1 

129 

A-l  1 

Flaps  30° 

82 

A-5 

130 

S/1 

83 

S/0 

133 

131 

S/0 

208 

84 

S/1 

132 

V/0 

Wing  Sweep  20° 

85 

V/3 

133 

s/1 

86 

S/0 

142 

135 

S/0 

212 

88 

A- 6 

100% 

89 

S/1 

Wing  Sweep  40' 

D 

91 

S/0 

146 

92 

S/1 

93 

A- 7 
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Appendix  E 

Equipment  Operating  Instructions 

To  give  the  most  useful  information  on  operating  the  equipment 
and  to  help  avoid  or  minimize  several  minor  complications,  the 
following  checklist  might  be  used  un.il  the  observer  is  familiar 
with  the  equipment  operation. 

One  hour  before  subject  run  time  (warm-up) 

1.  Plug  motor  speed  control  and  Timers  2  and  3  power  into 
Sola  transformer  (most  critical  sources  to  control), 

2.  Plug  in  Ester! ine-Angus,  Timer  1  (2  plugs).  Tally  reader, 
and  Control  Chassis  into  11 5v  a-c  (5  connections  to  115v). 

3.  Plug  Sola  transformer  into  11 5v  a-c.  (Be  sure  Timer  1 
is  plugged  in  before  motor  control  is  on  or  airspeed 
will  oegin  to  run  if  Panel  1  is  connected.) 

(1  connection  to  1 1 5v  a-c) 

4.  Plug  400  cycle  connection  into  wall  outlet. 

Fifteen  minutes  before  subject  run  time 

5.  Start  400  cycle  generator 

Restart  Sequence 

1.  Hunter  Timer  #2  Interval,  ON 

2.  Ester! ine-Angus  Recorder,  OFF 

3.  Airspeed,  Reset 

4.  Timing,  Set 

5.  Hunter  #3,  0N_ 

6.  Tape,  Check 

7.  Airspeed,  Run 

8.  Hunter  #2,  GW 

9.  Esterl ine-Angus,  ON 

in.  Panel  Airspeed  Counter,  Recheck 
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Appendix  F 
Recording  Equipment 


Table  XII 

Esterline- Angus  Channel  Functions 
Channel  Function 


1 

Brake 

2 

Throttle  Idle 

3 

^  73% 

4 

rv  80% 

5 

90% 

5 

100% 

7 

110% 

8 

— 

9 

Nose  Up 

10 

Nose  Down 

11 

Gear  Down 

12 

Gear  Up 

13 

Wing  Sweep  20° 

14 

Winy  Sweep  40° 

15 

Wing  Sweep  60° 

16 

Flaps  0° 

17 

Flaps  30° 

18 

Flaps  60° 

19 

Altitude  Reference 

20 

Time  Reference  (Sec) 
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INDIANAPOLIS,  IND.,  U.S.A,  CHART  NO.  t720-X 


Fig.  14 

Airspeed  -Indicator  Xircuit 
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Throttle  Circuit 


Fig.  18 

Flap  Switch  Circuit 
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(B54) 


tz: 


BRAKE 

SWITCH 


RELAY  TREE 
CODE  NO  17  , 


PIN  PLUG 


BRAKE  LITE  CIRCUIT 


TALLY  CONTROL 
CHASSIS  TB 
EXT,  FV?D. 


(A48) 
-  V 


+28V 


+28V- 


A 

A 


-29  - 

(C32) 


291 


6 


PIN  PLUG 
a  JACK 


JONEsW 


(C32) 


? 


BRAKE 

LITE 


HOLD  THRU 
RESET  RELAY 
(SEE  A/C  CIRCUIT) 


WING  SWEEP  CIRCUIT 


12  nN 
JONES  NOS 


Fig.  19 

Brake  Lite  and 
Wing  Sweep  Circuits 
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ESTERLINE  REFERENCE  CIRCUITS 
ALTIMETER  REFERENCE 

UNUSED  CHANNEL. 
(19) 

14 

ALTIMETER  RELAY 


TIME  REFERENCE 


#  NECESSARY  EXTRA  WIRES  ARE  IN  CABLES  FROM  A266C32 


Fig.  21 

Esterline  Reference 
Circuits 
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Fig.  22 

Tally  Timer  Circuit 
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